Recovering Joint Probability of Discrete Random Variables From Pairwise Marginals

Learning the joint probability of random variables (RVs) is cornerstone statistical signal processing and machine learning. However, direct nonparametric estimation for high-dimensional in general impossible, due to curse dimensionality. Recent work has proposed recover mass function (PMF) an arbitrary number RVs from three-dimensional marginals, leveraging algebraic properties low-rank tensor decomposition (unknown) dependence among RVs. Nonetheless, accurately estimating marginals can still be costly terms sample complexity, affecting performance this line practice sample-starved regime. Using also involves challenging problems whose tractability unclear. This puts forth a new framework learning PMF using only pairwise which naturally enjoys lower complexity relative third-order ones. A coupled nonnegative matrix factorization (CNMF) developed, its recovery guarantees under various conditions are analyzed. Our method features Gram--Schmidt (GS)-like algorithm that exhibits competitive runtime performance. The shown provably up bounded error finite iterations, reasonable conditions. It recently economical expectation maximization (EM) improve upon GS-like algorithm's output, thereby further lifting accuracy efficiency. Real-data experiments employed showcase effectiveness.